Information processing device and recording method

ABSTRACT

An information processing device includes: a detection unit that detects that an application enabling a call with another information processing device over a network occupies a microphone or a speaker; and a recording unit that starts recording of audio data output to the speaker and audio data input from the microphone in the case where the application detects that the microphone or the speaker is occupied.

TECHNICAL FIELD

The present disclosure relates to an information processing device and arecording method.

BACKGROUND

In recent years, the opportunities for teleworking have increased, andalong with this a video conference has also been held frequently.Regarding this type of video conference, there has been proposed asystem that records audio during the conference and records video datadisplayed on a display unit in order to check the contents of theconference after the end thereof and to convey the contents of theconference to those who could not attend the conference.

SUMMARY

According to the first aspect of the present disclosure, there isprovided an information processing device including: a detection unitthat detects that an application enabling a call with anotherinformation processing device over a network occupies a microphone or aspeaker; and a recording unit that starts recording of audio data outputto the speaker and audio data input from the microphone in the case ofdetecting that the application occupies the microphone or the speaker.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram illustrating an example of ahardware configuration of an information processing device according toan embodiment of the present disclosure.

FIG. 2 is a functional block diagram illustrating an example of arecording function among various functions of the information processingdevice according to an embodiment of the present disclosure.

FIG. 3 is a diagram illustrating an example of a software configurationrelated to the recording function among various functions of theinformation processing device according to an embodiment of the presentdisclosure.

FIG. 4 is a flowchart illustrating an example of a processing procedureof a recording method in an embodiment of the present disclosure.

DETAILED DESCRIPTION

In certain audio or video recording systems for a video conference, aspecific user who set the video conference has the authority to recordaudio or the like, and therefore each participant could not arbitrarilyrecord audio or the like.

In addition, some communication apps such as Microsoft Teams (registeredtrademark) and ZOOM (registered trademark), which have become popular inrecent years, have a recording function. An input operation for startingthe recording function, however, differs depending on the application.Therefore, when a conference or the like is held using an unfamiliarapplication, the start operation of audio recording or the like has beenperformed smoothly in some cases. In such a meeting application, theuser needs to perform an input operation for starting or stopping theaudio recording or the like and therefore is likely to forget the audiorecording.

In view of the above, it would be desirable to provide an informationprocessing device and a recording method capable of increasing theconvenience of recording.

An embodiment of an information processing device and a recording methodaccording to the present disclosure will be described below withreference to appended drawings. Examples of the information processingdevice include laptop PCs, desktop PCs, tablet terminals, smartphones,and the like.

FIG. 1 is a schematic configuration diagram illustrating an example of ahardware configuration of an information processing device 10 accordingto an embodiment of the present disclosure. As illustrated in FIG. 1,the information processing device 10 includes, for example, a centralprocessing unit (CPU) 11, a main memory 12, a storage unit 13, amicrophone (hereinafter, referred to as “mic”) 14, a speaker 15, anexternal interface 16, a communication unit 17, an input unit 18, adisplay unit 19, and the like. These units are connected to each otherdirectly or indirectly via a bus and perform various processes incooperation with each other. In addition, the information processingdevice 10 may be equipped with a camera or the like.

The CPU 11 controls the entire information processing device 10 by anoperating system (OS) stored in the storage unit 13 connected to the CPU11 via the bus, and performs various processes by executing variousprograms stored in the storage unit 13.

The main memory 12 is composed of writable memories such as a cachememory and a random access memory (RAM) and is used as a work area forreading an execution program of the CPU 11 and for writing dataprocessed by the execution program.

The storage unit 13, which is, for example, a read only memory (ROM), ahard disk drive (HDD), a flash memory, or the like, stores an OS forcontrolling the entire information processing unit 10 such as, forexample, Windows (registered trademark), an iOS (registered trademark),or an Android (registered trademark), various device drivers foroperating peripheral devices and the like in hardware, variousapplication software (hereinafter, simply referred to as“applications”), and various data and files. In addition, the storageunit 13 stores programs for implementing various processes and variousdata required for implementing various processes.

The mic 14, which runs on the basis of a command from the CPU 11,converts an acquired sound into an electric signal and outputs theelectric signal.

The speaker 15, which runs on the basis of a command from the CPU 11,converts an electric signal into sound and outputs the sound.

The mic 14 and the speaker 15 may be mounted on the informationprocessing device 10 or may be configured so as to be connected ordisconnected via an external interface 16 or a short-rangecommunication, which will be described later, or the like.

The external interface 16 is an interface for connecting to an externaldevice. Examples of the external device include an external monitor, aUSB memory, an external HDD, and the like. In the example illustrated inFIG. 1, only one external interface is illustrated, but a plurality ofexternal interfaces may be included.

The communication unit 17 functions as an interface for connecting to anetwork to communicate with other devices to send and receiveinformation.

The input unit 18 is a user interface for a user to give instructions tothe information processing device 10, such as a keyboard, a mouse, and atouch panel.

The display unit 19 has a display screen composed of, for example, aliquid crystal display (LCD), an organic electro luminescence (EL), orthe like, and displays the results of an application or the likeexecuted by the information processing device 10.

FIG. 2 is a functional block diagram illustrating an example of arecording function among various functions of the information processingdevice 10 according to this embodiment. As an example, a series ofprocesses for implementing the various functions described later isstored in the storage unit 13 in the form of a program, and the CPU 11reads the program into the main memory 12 and performs informationprocessing and arithmetic processing to implement various functions. Theprogram may be pre-installed in the storage unit 13, provided in a stateof being stored in another computer-readable storage medium, ordistributed via a wired or wireless communication means. Thecomputer-readable storage medium includes a magnetic disk, anopto-magnetic disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and thelike.

As illustrated in FIG. 2, the information processing device 10 includesan application (hereinafter, referred to as “communication app”) 20 thatenables calls via a network with other information processing devicesand a recording system 30.

The communication app 20 is, for example, an application forimplementing communication using audio and video on the Web. Forexample, there are Microsoft Teams (registered trademark), ZOOM(registered trademark), LINE (registered trademark), SKYPE (registeredtrademark), and the like.

Although FIG. 2 illustrates the case of having three communication apps20 a, 20 b, and 20 c, at least one communication app may be provided. Inaddition, this communication app 20 may run on a cloud via a web browserand does not necessarily have to be installed on the informationprocessing device 10.

The recording system 30 has a detection unit 31, a recording unit 32,and a file creation unit 33.

The detection unit 31 detects that the communication app 20 occupies themic 14 (see FIG. 1) or the speaker (see FIG. 1).

For example, if the detection unit 31 detects that any applicationoccupies the mic 14 or the speaker 15 and further the application is apre-registered communication app 20, the detection unit 31 may detectthat the communication app 20 occupies the mic 14 or the speaker 15.

For example, the detection unit 31 has a registration list (so-called“whitelist”) in which the application names of the communication apps 20for which data are recorded are previously registered. Then, in the casewhere the detection unit 31 detects that any application occupies themic 14 or the speaker 15, the detection unit 31 determines whether theapplication name of the application is registered in the registrationlist. If the application name is registered in the registration list,the detection unit 31 detects that the communication app 20 occupies themic 14 or the speaker 15.

In addition, the detection unit 31 may have a registration list(so-called “blacklist”) in which applications other than those for whichdata are recorded are registered, instead of the above whitelist. Inthis case, unless the application that occupies the mic 14 or thespeaker 15 is registered in the registration list (blacklist), thedetection unit 31 detects that the application for which data arerecorded occupies the mic 14 or the speaker 15. For example, theblacklist contains microphone volume setting apps and audio assistantapps. By using the registration list as a blacklist, data recording isable to be started even with an unknown communication app.

Note that the registration list to be referenced may be differentdepending on whether the mic 14 is occupied or the speaker 15 isoccupied. For example, the application may be determined to be anapplication for which data are recorded by using the blacklist in whichapplications other than those for which data are recorded are registeredin the case where the occupation of the mic 14 is detected or by usingthe whitelist in which applications for which data are recorded areregistered in the case where the occupation of the speaker 15 isdetected. This is because there are generally more applications thatoccupy the speaker 15 than those that occupy the mic 14. In addition,the application names registered in these registration lists may bethose able to be customized by a user.

The above “state in which the mic 14 is occupied” means a state in whichthe communication app 20 is permitted to use the mic 14, and morespecifically, a state in which the OS gives the application the right touse the mic 14 (occupancy right). For example, the state of occupyingthe mic 14 does not mean a state in which the function of the mic 14 istemporarily switched from OFF to ON in the application by a user's inputoperation for muting and unmuting during a web conference using thecommunication app 20.

In addition, the state in which the communication app occupies thespeaker 15 means a state in which the communication app 20 is permittedto use the speaker 15. More specifically, the state means a state inwhich the OS gives the communication app 20 the right to use the speaker15 (occupancy right).

Moreover, the detection unit 31 detects that the communication app 20that has occupied the mic 14 or the speaker 15 released the mic 14 orthe speaker 15.

In the above, the release of the mic 14 means a state in which thecommunication app 20 released the right to use the mic 14, and morespecifically, a state in which the right to use the mic 14 that has beengiven to the communication app 20 by the OS is released. For example,the state in which the mic 14 is released does not mean a state in whichthe function of the mic 14 is temporarily switched from ON to OFF in theapplication by a user's input operation for muting and unmuting during aweb conference using the communication app 20.

In addition, the release of the speaker 15 means a state in which thecommunication app 20 released the right to use the speaker 15, and morespecifically, a state in which the right to use the speaker 15 that hasbeen given to the communication app 20 by the OS is released.

The recording unit 32 starts to record audio data output from thespeaker 15 and audio data input from the mic 14 in the case where thedetection unit 31 detects that the communication app 20 occupies the mic14 or the speaker 15. The recording unit 32 records only audio dataoutput from the speaker 15, for example, in the case where it isdetected that the communication app 20 occupies the speaker 15 and wherethe mic 14 is turned off.

In addition, the recording unit 32 may start to record video data outputto the display unit 19 (see FIG. 1). The video data to be recorded maybe, for example, video data displayed in a window of the applicationoccupying the mic 14 or the speaker 15. This enables a user to avoidrecording even video data unrelated to the video conference in the casewhere the user has multiple windows open.

In addition, the recording unit 32 stops the recording of audio data inthe case of detecting that the mic 14 or the speaker 15 that has beenoccupied by the communication app 20 is released. Moreover, therecording unit 32 also stops the recording of video data in the casewhere the video data has been recorded.

The file creation unit 33 stores the data recorded by the recording unit32 as a data file in the storage unit 13. At this time, the data filemay be a data file created by merging audio data output to the speaker15 and audio data input from the mic 14. Furthermore, in the case wherevideo data has been also recorded, the data file may be a data filecreated by merging the audio data and the video data. Note that, inaddition to merging audio data and video data, these data may be storedas individual files in the storage unit 13. Thus, the data storage modeis not particularly limited.

As for the data merging technique, various known techniques areapplicable as appropriate.

The file creation unit 33 acquires the application name of thecommunication app 20 that occupies the mic 14 or the speaker 15 and putsthe acquired application name in the title or metadata of the data filestored in the storage unit 13. Moreover, information about the date andtime when the recording is performed may be further included in thetitle or metadata.

Furthermore, the file creation unit 33 may acquire the window title (forexample, the meeting name) of the communication app 20 that occupies themic 14 or the speaker 15, instead of or in addition to the applicationname, and may put the acquired window title in the title or metadata ofthe data file stored in the storage unit 13.

The above application name and window title can be acquired from, forexample, an OS, calendar information, or the like.

The storage unit 13 stores data recorded by the recording unit 32, andthe data file name or metadata thereof includes the title and windowtitle of the communication app that performed the recording. Thisenables the user to easily check when and which meeting the datacorresponds to.

Subsequently, an example of a software configuration for implementingthe recording method of the information processing device 10 describedabove will be described with reference to FIG. 3. FIG. 3 is a diagramillustrating an example of the software configuration related to therecording function among various functions of the information processingdevice 10 according to this embodiment.

As illustrated in FIG. 3, the information processing device 10 includesa communication app 20 (20 a to 20 c), a recording application(hereinafter, referred to as “recording app”) 40, and a media system 50.

The communication app 20 is application software that runs on the OS,and as mentioned above, it is an application that enables calls via anetwork with other information processing devices.

The recording app 40 is application software that runs on the OS and isan application for implementing the functions of the recording system 30illustrated in FIG. 2. For example, the recording app 40 is a programfor implementing the functions of the detection unit 31, the recordingunit 32, and the file creation unit 33 illustrated in FIG. 2.

The media system 50 is an OS component that controls audio and, forexample, controls audio output in response to audio-related requestsfrom various applications.

Subsequently, processing implemented by the recording app 40 will bedescribed with reference to FIG. 4. FIG. 4 is a flowchart illustratingan example of a processing procedure of the recording method in thisembodiment. In the following processing procedure, description will bemade by way of example of using a registration list (so-called“whitelist”) in which an application for performing recording isregistered.

First, when the information processing device 10 is started, therecording app 40 is started and an event subscription is output to themedia system 50 (SA1). This means that, for example, in the case where arequest for occupying the mic 14 (see FIG. 3) or the speaker 15 (seeFIG. 3) is input from any application to the media system 50, anotification thereof is requested.

When the request for occupying the mic 14 or the speaker 15 is inputfrom any application running on the OS, the media system 50 notifies therecording app 40 of the occupation of the mic 14 or the speaker 15(SA2). At this time, the media system 50 also outputs the name of theapplication that issued the request for occupying the mic 14 or thespeaker 15 together with the occupation notification.

Upon receiving the notification of occupying the mic 14 or the speaker15, the recording app 40 determines whether the application name in thenotification output from the media system 50 together with theoccupation notification corresponds to the communication app 20registered in the registration list (“whitelist”) (SA3). Unless theapplication name is registered in the registration list as a result(SA3: NO), a standby state is maintained until a notification ofoccupying the mic 14 or the speaker 15 is received again, withoutperforming recording. If the application name is registered in theregistration list (SA3: YES), the occupation of the mic 14 or thespeaker 15 by the communication app 20 is detected and recording isstarted (SA4).

Upon starting the recording, the recording app 40 notifies the mediasystem 50 of the start of recording (SA5). At this time, the recordingapp 40 may display a message such as “Recording starts” on the displayunit 19 to notify the user of the start of audio recording.

Subsequently, upon input of a request for releasing the mic 14 or thespeaker 15 to the media system 50 from the application that has occupiedthe mic 14 or the speaker 15 due to an end of a conference or a call orthe like, the media system 50 notifies the recording app 40 of therelease of the mic 14 or the speaker 15 (SA6). Furthermore, in additionto the notification of the release, the media system 50 outputsinformation (for example, an application name) on the application thathas issued the request for releasing the mic 14 or the speaker 15 to therecording app 40.

Upon receiving the notification of releasing the mic 14 or the speaker15, the recording app 40 determines whether the mic 14 or the speaker 15is released from the occupation by the communication app 20 that hasbeen a target of recording (SA7). If the determination is negative as aresult (SA7: NO), the audio recording is continued as it is. On theother hand, if the determination is positive (SA7: YES), the release ofthe mic 14 or the speaker 15 by the communication app 20 is detected andthe recording is stopped (SA8). Upon stopping the recording, therecording app 40 notifies the media system 50 of the recording stop(SA9). At this time, the recording app 40 may display a message such as“Recording ends” on the display unit 19 to notify the user of the end ofaudio recording.

Upon the end of recording as described above, for example, the storageunit 13 stores data created by merging, for example, audio data andvideo data of a video conference or the like as a data file. Moreover,for example, the file name and metadata of the data file includesattribute information such as the application name, the window title,and the meeting date and time of the communication app 20 that hasoccupied the mic 14 or the speaker 15.

Then, the above-mentioned processing of SA2 to SA9 is repeated until theinformation processing device 10 is shut down, by which the audio dataand video data of the conference are automatically stored in the storageunit 13 every time a video conference or the like is performed.

As described hereinabove, according to the information processing device10 and the recording method of this embodiment, the recording of theaudio data input to the mic 14, the audio data output to the speaker 15,and the like is started upon detecting that the communication app 20 hasoccupied the mic 14 or the speaker 15, while the recording of the audiodata and the like is stopped upon detecting that the communication app20 has released the mic 14 or the speaker 15. Thereby, the audio data ofa conference or the like are automatically recorded without a user'sinput operation for audio or video recording. This enables an increasein convenience. Furthermore, audio recording for audio data or the likeis able to be performed locally, and therefore the user does not have toworry about other participants.

Furthermore, the recording of audio data and the like is starteddepending on whether the speaker 15 has been occupied, thereby enablinguse for recording in the case of attending a conference without usingthe mic 14 such as, for example, in the case of providing a seminar tomultiple participants via the Web.

Moreover, recorded audio data is digital audio data output from the mic14 and digital audio data output from the media system 50 to the speaker15 as illustrated in FIG. 3, by which the recorded audio data can berecorded with higher sound quality and the data volume of the audio datacan be reduced as compared with the case of recording analog data.

Furthermore, in the case of recording video data at the same time asaudio data, there is recorded the video data displayed in the window ofthe communication app 20 that occupies the mic 14 or the speaker 15.This makes it possible to avoid recording video data other than those ofa conference, for example, in the case of working with other windowsdisplayed.

In addition, the title or metadata of the data file stored in thestorage unit 13 includes the application name and window title (forexample, a meeting name) of the communication app 20 used for holding aconference or the like, thereby enabling the user to easily search forand find a desired data file.

In the information processing device 10 and the recording methodaccording to this embodiment, audio data and video data areautomatically recorded as described above, which may squeeze availabledata space of the storage unit 13. Therefore, for example, in the casewhere a predetermined data volume is exceeded, an alarm may be issued tothe user, or a function may be provided to automatically delete olddata. For example, since the volume of video data is larger than thevolume of audio data, image frames may be extracted from old video dataat predetermined time intervals and only the extracted image frames maybe recorded in the case where the volume of video data exceeds apredetermined value. Specifically, the compression level of the videodata is adjusted to be higher for older video data.

Although the present disclosure has been described above using theembodiments, the technical scope of the present disclosure is notlimited to the scope described in the above embodiments. Variousmodifications or improvements can be made to the above embodimentswithout departing from the gist of the invention, and the modified orimproved forms are also included in the technical scope of the presentdisclosure. Further, the above embodiments may be combined asappropriate.

In addition, the procedure for the recording method described in theabove embodiment is also an example, and unnecessary steps may bedeleted, new steps may be added, and the processing order may be changedwithout departing from the gist of the present disclosure.

1. An information processing device comprising: a microphone; a speaker;and a processor coupled to a memory, the processor being programmed toexecute an operating system and to further: detect whether anapplication running on the operating system and which is conducting acall with another information processing device over a network occupiesthe microphone or the speaker, and upon detecting that the applicationoccupies the microphone or the speaker, begin recording of audio dataoutput to the speaker and audio data input from the microphone.
 2. Aninformation processing device comprising: a detection unit that detectsthat an application enabling a call with another information processingdevice over a network occupies a microphone or a speaker; and arecording unit that starts recording of audio data output to the speakerand audio data input from the microphone in the case of detecting thatthe application occupies the microphone or the speaker.
 3. Theinformation processing device according to claim 2 wherein: thedetection unit detects that the application has released the microphoneor the speaker; and the recording unit stops the recording of the audiodata in the case of detecting that the microphone or the speaker isreleased.
 4. The information processing device according to claim 2,wherein the recording unit starts recording of video data output to adisplay unit in the case of detecting the occupation of the microphoneor the speaker and stops the recording of the video data in the case ofdetecting a release of the microphone or the speaker.
 5. The informationprocessing device according to claim 4, wherein the recording unitrecords the video data displayed in a window of the application thatoccupies the microphone or the speaker.
 6. The information processingdevice according to claim 2, further comprising a file creation unitthat acquires a name of the application that occupies the microphone orthe speaker and puts the acquired application name in a file name orputs metadata of the data recorded by the recording unit in a file. 7.The information processing device according to claim 2, furthercomprising a file creation unit that acquires a window title of theapplication that occupies the microphone or the speaker and puts theacquired window title in a file name or puts metadata of the datarecorded by the recording unit in a file.
 8. A computer-implementedmethod for recording, comprising: detecting that an application enablinga call with another information processing device over a networkoccupies a microphone or a speaker; and starting recording of audio dataoutput to the speaker and audio data input from the microphone in thecase of detecting that the application occupies the microphone or thespeaker.
 9. The recording method according to claim 8 wherein furthercomprising: detecting that the application has released the microphoneor the speaker; and stopping the recording of the audio data in the caseof detecting that the microphone or the speaker is released.